fix(agent): defer intake and provider web tools by kojiwakayama · Pull Request #3536 · veryfront/veryfront-code

kojiwakayama · 2026-08-10T14:52:49Z

Summary

defer form_input from the hosted bootstrap while keeping it searchable and checkpoint-restorable
expose configured provider-native web tools as schema-free deferred capabilities, then attach the native provider schema only after tool_search activates one
keep deferred names and schemas out of the initial provider request and align hosted steering refresh with the new bootstrap
add end-to-end activation, authorization, privacy, budget, and durability regression coverage

Relates to veryfront/veryfront-issue-inbox#354.

Verification

deno test --no-check --allow-all --parallel src/agent/runtime src/agent/hosted (932 passed)
focused post-rebase runtime, hosted, durability, and stream suite (77 passed, 132 steps)
deno task verify:quick
deno task test:unit (3,785 passed, 0 failed, 1 ignored)
deno task audit (0 vulnerabilities)

Summary by CodeRabbit

New Features
- Provider-native tools are now discovered dynamically and exposed only when available in the current runtime step.
- Deferred tools, including web search and form input, can be searched and activated when needed.
- Tool inventories now provide deterministic names and descriptions.
Bug Fixes
- Prevented unavailable provider tools from being called prematurely.
- Improved tool visibility, authorization, checkpoint restoration, and execution validation.
Documentation
- Updated guidance for deferred tools and provider-native tool discovery.
- Updated provider-native tool source references in the API documentation.

coderabbitai · 2026-08-10T14:53:28Z

Warning

Review limit reached

@kojiwakayama, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 14 seconds

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e98f2492-57d8-47ac-af39-2f7304c437a7

📥 Commits

Reviewing files that changed from the base of the PR and between 90836a8 and 5fbee85.

📒 Files selected for processing (1)

docs/architecture/28-model-driven-tool-discovery.md

📝 Walkthrough

Walkthrough

The change separates bootstrap tools from deferred tools, adds deterministic provider-native exposure definitions, filters provider execution by current visibility, and expands tests and documentation for activation, authorization, prompts, budgets, and runtime execution.

Changes

Deferred provider-native tool exposure

Layer / File(s)	Summary
Bootstrap visibility and deferred state `src/agent/runtime/tool-exposure.ts`, `src/agent/hosted/...`, `src/agent/runtime/tool-exposure.test.ts`, `tests/chat/...`	`load_skill` remains bootstrapped. `form_input` becomes deferred. Tests cover assembly, activation, restoration, capacity, pruning, and streaming visibility.
Provider-native inventory and runtime filtering `src/agent/runtime/provider-native-tool-inventory.ts`, `src/agent/runtime/agent-runtime-step.ts`, `src/agent/runtime/index.ts`, `src/agent/runtime/...test.ts`	Provider-native tools receive deterministic schema-free definitions. Runtime authorization and execution use only currently visible provider tools.
Runtime validation and documentation contracts `src/agent/runtime/tool-exposure-runtime.test.ts`, `docs/architecture/...`, `docs/guides/...`, `docs/api-reference/...`, `tests/docs/...`	Tests verify hidden prompt tools, deferred `web_search` activation, provider execution, tool-call statuses, and final responses. Documentation describes deferred provider discovery and updated source links.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Model
  participant AgentRuntime
  participant ToolExposure
  participant Provider
  Model->>AgentRuntime: request available tools
  AgentRuntime->>ToolExposure: resolve visible tools
  ToolExposure-->>AgentRuntime: return visible names
  AgentRuntime-->>Model: expose visible tools
  Model->>AgentRuntime: call tool_search
  AgentRuntime->>ToolExposure: activate deferred provider tool
  ToolExposure-->>AgentRuntime: update exposure state
  AgentRuntime->>Provider: execute visible provider tool
  Provider-->>AgentRuntime: return tool result
  AgentRuntime-->>Model: provide result and final response

Possibly related PRs

veryfront/veryfront-code#3393: Extends the same provider-native inventory and deferred exposure behavior.
veryfront/veryfront-code#3433: Changes deferred visibility for the same load_skill tool.
veryfront/veryfront-code#3504: Changes the shared deferred tool-discovery system.

Suggested reviewers: kwakayama

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main changes: deferring intake and provider web tools in the agent runtime.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/issue-354-deferred-bootstrap-tools

_{Comment @coderabbitai help to get the list of available commands.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4be05673da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/architecture/28-model-driven-tool-discovery.md`:
- Around line 81-93: Update the Execution gate description to remove “unchanged”
and explicitly describe both checks: prepareExecution revalidates remote
allowance through isRemoteToolNameAllowed, and provider-native exposure is
intersected with the current configured provider tools and selected-model
support before each step.
- Around line 41-45: Update the measurement section around the “initially
exposed = 1” metric to state that the hi fixture authorizes exactly 64 generated
tools and does not authorize load_skill, while preserving the explanation that
tool_search is the sole initially exposed tool.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b2637b89-46a2-427e-8e48-e600b485c248

📥 Commits

Reviewing files that changed from the base of the PR and between 4be0567 and 90836a8.

📒 Files selected for processing (6)

docs/architecture/28-model-driven-tool-discovery.md
docs/guides/agents.md
docs/guides/tools.md
src/agent/runtime/tool-exposure-runtime.test.ts
tests/chat/agent-runtime-streaming.test.ts
tests/docs/guide-content.test.ts

🚧 Files skipped from review as they are similar to previous changes (1)

src/agent/runtime/tool-exposure-runtime.test.ts

fix(agent): defer intake and provider web tools

4be0567

kojiwakayama requested a review from kwakayama as a code owner August 10, 2026 14:52

chatgpt-codex-connector Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread src/agent/runtime/provider-native-tool-inventory.ts

kojiwakayama added 3 commits August 10, 2026 16:59

docs(agent): align deferred tool discovery guides

ead5081

test(agent): satisfy exposure test typecheck

15173d6

test(agent): update deferred streaming exposure contract

90836a8

coderabbitai Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread docs/architecture/28-model-driven-tool-discovery.md Outdated

Comment thread docs/architecture/28-model-driven-tool-discovery.md

docs(agent): clarify deferred exposure evidence

5fbee85

kojiwakayama added this pull request to the merge queue Aug 10, 2026

Merged via the queue into main with commit 269df3f Aug 10, 2026
31 checks passed

kojiwakayama deleted the fix/issue-354-deferred-bootstrap-tools branch August 10, 2026 15:46

kojiwakayama mentioned this pull request Aug 10, 2026

Release v0.1.1228 #3539

Closed

kwakayama mentioned this pull request Aug 10, 2026

Release v0.1.1228 #3554

Merged

Navigation Menu

Uh oh!

fix(agent): defer intake and provider web tools - #3536

fix(agent): defer intake and provider web tools#3536
kojiwakayama merged 5 commits into
mainfrom
fix/issue-354-deferred-bootstrap-tools

kojiwakayama commented Aug 10, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Aug 10, 2026 •

edited

Loading

Review limit reached

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

kojiwakayama commented Aug 10, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

kojiwakayama commented Aug 10, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 10, 2026 •

edited

Loading